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Objective 

•  Problem: 

•  Chief  of  Staff  of  the  Air  Force  (CSAF) 

•  Air  Force  Studies  and  Analysis  Agency  (AFSAA) 

•  Request  an  effective  &  efficient  method  to  reduce  the 
variance  in  the  results  of  TF1UNDER 

•  Answer: 

•  Control  Variates 

•  Common  Random  Numbers 

•  Antithetic  Variates 


THUNDER 


•  Air  Force  Standard  Analysis  Toolkit  (AFSAT) 

•  Campaign-level  joint  military  operations 

•  Written  in  CACI’s  SIMSCRIPT  II.5 

•  English  like  programming  language 

•  >1,500  source  files 

•  >360,000  lines  of  code 

•  254  random  inputs 
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Common  Random  Numbers 


•  Synchronization 


Random  Inputs 


Config  1 


Config  2 


•Problem  with  current  RNG 


254  random  variates  and  only  10  streams 


Random  Number  Generator 


MRG32k3a 
Developed  by 

•  Pierre  L’Ecuyer 

•  U.  de  Montreal 

•  Richard  Simard 

•  U.  de  Montreal 

•  E.  Jack  Chen 

•  BASF  Corp. 

•  W.  David  Kelton 

•  U.  of  Cincinnati 


Linear  Congruential 
Generator 

•  period  1.6  x  107  to  2.8  x 
1014  elements 

Combined  Multiple 
Recursive  Generator 

•  1.8  x  1019  streams 

•  1.7  x  1038  elements 
Programmed  in  C  and  Java 

•  Streams,  Antithetics,  Seeds 


Control  Variates  Flow  Chart 
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Conclusions 


Measures  Of  Effectiveness 


20  MOEs 

•  Flown  per  Planned 
Sortie 

•  Losses  per  Sortie 

•  Square  Miles  Gained 


4  Levels  of  Aggregation 

•  Force 

•  Base 

•  Squadron 

•  System 


Experimental  Runs 


•  8  separate  run  configurations 

•  30  Replications  each 

•  240  T otal  Replications 

•  ASC  Major  Shared  Resource  Center 


RNG 

SYNCHRONIZED 

ANTITHETIC 

COLLECT  CV 

CONFIGURATION 

SET 

(NEW/OLD) 

(YES/NO) 

(YES/NO) 

(YES/NO) 

(ORIGINAL/MODIFIED) 

A 

OLD 

NO 

NO 

YES 

ORIGINAL 

B 

NEW 

NO 

NO 

NO 

ORIGINAL 

C 

NEW 

NO 

NO 

YES 

MODIFIED 

D 

NEW 

YES 

NO 

YES 

ORIGINAL 

E 

NEW 

YES 

NO 

YES 

MODIFIED 

F 

NEW 

YES 

YES 

YES 

ORIGINAL 

G 

OLD 

NO 

NO 

NO 

ORIGINAL 

H 

NEW 

YES 

NO 

NO 

ORIGINAL 

Common  Random  Numbers 


•  Reduction 
•  10  of  19 


•  No  patterns 


MOE 

BASE 

HALFWIDTH 

SYNCHRONIZED 

HALFWIDTH 

PERCENT 

CHANGE 

A-1 0  LPS 

0.001940153 

0.001590769 

-18.01% 

A-6  LPS 

0.005187968 

0.003856901 

-25.66% 

BLUE  FPS 

0.010247512 

0.008539409 

-16.67% 

BLUE  LPS 

0.000770395 

0.000659791 

-14.36% 

BLUE  SMG 

401.9576399 

357.5307451 

-11.05% 

DHAHRAN  FPS 

0.029904287 

0.027359177 

-8.51% 

F-15  LPS 

0.001401777 

0.001609386 

14.81% 

FA-18  FPS  total 

0.016106877 

0.01672975 

3.87% 

FA-18  FPS  DCA 

0.005501239 

0.004594474 

-16.48% 

MIG-23  LPS 

0.032332763 

0.024092911 

-25.48% 

MIG-29  LPS 

0.027652343 

0.030965238 

1 1 .98% 

MIRAGE  FPS  total 

0.017106674 

0.025227701 

47.47% 

MIRAGE  FPS  INT 

0.029598399 

0.030482717 

2.99% 

MIRAGE  LPS 

0.01957014 

0.015011175 

-23.30% 

MUDAYSIS  FPS 

0.016871397 

0.017668922 

4.73% 

RED  FPS 

0.006272345 

0.006833599 

8.95% 

RED  LPS 

0.005909479 

0.007268262 

22.99% 

RIYADH  FPS 

0.006359085 

0.005819308 

-8.49% 

SHAIBAH  FPS 

0.034021777 

0.040368895 

18.66% 

Control  Variates 


MOE 

REPLICATION 

HALFWIDTH 

cv 

HALFWIDTH 

PERCENT 

CHANGE 

EQUIVALENT 

REPLICATIONS 

A-10LPS 

0.00075445 

0.000159564 

-78.85% 

607 

A-6LPS 

0.003489524 

0.002120285 

-39.24% 

77 

BLUE  FPS 

0.009062548 

0.007242324 

-20.09% 

46 

BLUE  LPS 

0.000561401 

0.00033589 

-40.17% 

79 

BLUE  SMG 

268.0705303 

236.1766719 

-11.90% 

38 

DHAHRAN  FPS 

0.031932469 

0.024634985 

-22.85% 

49 

F-lll  LPS 

0.001710663 

0.000817956 

-52.18% 

121 

F-15  LPS 

0.000969011 

0.000627455 

-35.25% 

68 

FA-18  FPS  total 

0.016933126 

0.015413409 

-8.97% 

36 

FA- 18  FPS  DCA 

0.003428121 

0.003311208 

-3.41% 

32 

MIG-23  LPS 

0.020114382 

0.020331007 

1.08% 

30 

MIG-29  LPS 

0.026351157 

0.022059499 

-16.29% 

42 

MIRAGE  FPS  total 

0.015716995 

0.013363553 

-14.97% 

41 

MIRAGE  FPS  INT 

0.022547304 

0.013958327 

-38.09% 

74 

MIRAGE  LPS 

0.014436042 

0.013046527 

-9.63% 

37 

MUDAYSIS  FPS 

0.013106485 

0.012044326 

-8.10% 

35 

RED  FPS 

0.004319957 

0.003268615 

-24.34% 

51 

RED  LPS 

0.005896852 

0.005145653 

-12.74% 

39 

RIYADH  FPS 

0.006054193 

0.004081502 

-32.58% 

63 

SHAIBAH  FPS 

0.022973304 

0.016907879 

-26.40% 

53 

Control  Variates 


MOE 

WEIGHT 

CONTROL 

MOE 

WEIGHT 

CONTROL 

A-10  LPS 

-0.0002 

AIR060  2 

MIG-23  LPS 

-6.6526 

ADF121 2 

A-6  LPS 

-0.0071 

AIR050 2 

0.1433 

AIR101 1 

-0.0759 

ISR000  4 

MIG-29  LPS 

0.6471 

AIR5281 

BLUE  FPS 

-0.0722 

AIR050 1 

-1.4314 

BSE050 1 

-0.0659 

AIR070  2 

-0.2311 

GRD095  1 

-0.0035 

AIR800 3 

-0.7147 

ISR000 4 

0.3282 

BSE400  1 

MIRAGE  FPS  total 

-0.3242 

ADF150 1 

46.8237 

PLA443  1 

0.5198 

AIR561 1 

0.6659 

UTL1041 

0.0855 

AIR602 1 

BLUE  LPS 

0.0101 

ADF1502 

MIRAGE  FPS  INT 

-3.3544 

ADF121 1 

BLUE  SMG 

-23825.1823 

ADF105 1 

7.9362 

ADF121 2 

40.5388 

AIR060  2 

0.0101 

AIR800 3 

3649.5981 

AIR101 2 

-1.0679 

BSE003 2 

-1408.6753 

Al  R5612 

106.4336 

PLA443  1 

-20765.0668 

AIR840 1 

MIRAGE  LPS 

-2.3615 

ADF121 1 

-27766.1967 

AIR840  2 

MUDAYSIS  FPS 

-0.2438 

ADF150 2 

-8446.2005 

BSE400  1 

-0.0089 

AIR800 3 

DHAHRAN  FPS 

-0.1779 

AIR050 1 

0.1621 

GRD095  2 

-0.2230 

AIR070  2 

RED  FPS 

0.6949 

BSE200  5 

0.9677 

BSE400  1 

RED  LPS 

0.0922 

ADF150 1 

F-111  LPS 

0.0213 

ADF1502 

-0.0629 

AIR070 2 

-0.0120 

AIR060 1 

0.6117 

UTL104 1 

-0.1221 

BSE200  5 

RIYADH  FPS 

-0.2501 

AIR007 1 

F-15  LPS 

0.0069 

AIR050 1 

-0.0395 

AIR101 1 

0.0019 

AIR050  2 

-0.0621 

AIR101 2 

FA-18  FPS  total 

0.0252 

AIR050 2 

-0.0505 

GRD095 1 

-0.0889 

AIR070  2 

SHAIBAH  FPS 

-0.1675 

AIR101 1 

-1.3057 

AIR8102 

1.3115 

BSE003  1 

FA-18  FPS  DCA 

1.4195 

ADF121 3 

0.0006 

AIR060  2 

CONTROL 

WEIGHT 

MODULE  PURPOSE 

SPECIFIC  DRAW  PURPOSE 

ADF105_1 

-23825.18 

Used  to  calculate  the  flight  groups  position,  altitude, 
speed,  average  location,  and  delivery  profile. 

Used  to  check  for  point  on  battlefield. 
.INTER VIS. ANGLE  -  (Distribution)  / 
RADIAN.  C 

AIR060  2 

40.54 

Manages  defensive  anti  tactical  ballistic  missile 
mission  detections  and  engagements. 

Used  in  setting  saturation  delay  on 
shooter.  (Distribution) 

AIR1012 

3649.60 

Sets  up  and  air  engagement  by  creating  and  air-to-air 
engagement  for  each  flight  to  be  used  for  storing 
computations. 

Used  in  determining  the  engagement 
probability,  (.prob. engage  <  U(0,1)) 

AIR5612 

-1408.68 

Determines  if,  having  arrived  at  the  target’ s  estimated 
coordinate,  the  flight  group  can  find  the  target. 

Determines  the  success  of  a  mid¬ 
course  update.  (U(0,1)<=  .best.prob) 

AIR840  1 

-20765.07 

Determines  the  number  of  target  elements  that  are 
destroyed  when  a  given  weapons  area  of  effect  fully 
or  partially  covers  a  target. 

Determine  whether  a  live  target  element 
is  hit.  (U(0,l)<.prob.hit.live) 

AIR840_2 

-27766.20 

Determines  the  number  of  target  elements  that  are 
destroyed  when  a  given  weapons  area  of  effect  fully 
or  partially  covers  a  target. 

Determine  whether  a  live  target  element 
is  hit.  (U(0,l)<.exp.live.hits) 

BSE400_1 

-8446.20 

Determine  s  the  availability  of  aircraft,  establishes  the 
munition  configurations,  allocates  fuels  and 
determines  whether  to  cancel  or  run  the  mission  as 

scheduled. 

Determines  the  cancellation  of  a  sortie 
due  to  degrade. 

(U(0, 1  )<eawp  .sortie,  degrade) 

A-10  LPS 
A-6  LPS 
BLUE  FPS 
BLUE  LPS 
BLUESMG 
DHAHRAN  FPS 
F-111  LPS 
F-15LPS 
FA-18  FPS  total 
FA-18  FPS  DCA 
MIG-23  LPS 
MIG-29  LPS 
MIRAGE  FPS  total 
MIRAGE  FPS  INT 
MIRAGE  LPS 
MUDAYSIS  FPS 
RED  FPS 
RED  LPS 
RIYADH  FPS 
SHAIBAH  FPS 
TIME 


Random  Number  Generator 


Original  vs. 

New  (not  synchronized) 


Original  vs. 

New  (synchronized) 


New  (not  synchronized)  vs. 
New  (synchronized) 


■0.02 


0.02-0.02 


0.02-0.02 


0.02 
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Conclusions 


•  Common  random  numbers 

•  Generally  synchronize 

•  Need  new  random  number  generator 

•  Will  not  always  result  in  reduction 

•  Control  variates 

•  Always  reduction 

•  Better  estimates  with  less  replications 

•  Good  insight 

•  Able  to  be  automated 

•  Both  methods  are  applicable  to  STORM 


QUESTIONS? 


Thank  You 

earl.bednar@rl.af.mil 


